Complementary Matching Pursuit Algorithms for Sparse Approximation

Sparse coding in a redundant basis is of considerable interest in many areas of signal processing. The problem generally involves solving an under-determined system of equations under a sparsity constraint. Except for the exhaustive combinatorial approach, there is no known method to find the exact solution for general dictionaries. Among the various algorithms that find approximate solutions, pursuit algorithms are the most well-known. In this paper, we introduce the concept of a complementary matching pursuit (CMP). Unlike the classical matching pursuit (MP), which selects the atoms in the signal space, the CMP selects the atoms in the coefficient space. On a conceptual level, the MP searches for ’the solution vector among sparse vectors’ whereas the CMP searches for ’the sparse vector among the solution vectors’. We assume that the observations can be expressed as pure linear sums of atoms without any additive noise. As a consequence of the complementary actions in the coefficient space, the CMP does not minimize the residual error at each iteration, however it may converge faster yielding sparser solution vectors than the MP. We show that when the dictionary is a tight frame, the CMP is equivalent to the MP. We also present the orthogonal extensions of the CMP and show that they perform the complementary actions to those of their classical matching pursuit counterparts.